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QN  THE  EVALUATION  QF  NOISE  SAMPLES 
Arnold  J„  T0  Siegert 


AtiflttiQi » 


Unless  the  primary  source  of  noise  is  one  of  those  whloh  are 
theoretically  tractable,  the  statistical  properties  of  the 
noise  have  to  be  inferred  from  sample s0  We  have  developed 
some  criteria  to  aid  in  the  decision  whether  a  sample  can 
reasonably  be  assumed  to  have  come  from  a  Gaussian  noise 
with  predetermined  parameters e 

^  \ 
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Introductlom  The  optimum  design  of  instruaents  euoh  as  aiming  devioee 
requires  the  knowledge  of  the  statistical  properties  of  the  noise  which 
will  accompany  the  signals  to  which  the  devices  react 0  Unless  the  primary 
source  of  this  noise  is  one  of  those  which  are  theoretically  completely 
tractable,  such  as  thermal  noise,  shot  effect,  etc„,  the  statistical 
properties  of  the  noise  must  be  obtained  empirically  from  samples0  Since 
the  size  of  the  sample  is  often  limited  by  the  cost  of  obtaining  samples, 
the  problem  arises  to  which  extent  the  statistical  properties  obtained 
from  a  given  sample  can  be  relied  uponc  This  problem  has  been  given 
precise  formulation  in  the  statistical  literature  for  samples  taken  from 
populations  consisting  of  discrete  elementso  Slnoe  the  elements  of  the 
sample  in  our  case  are  the  observed  values  of  y(t )  in  a  time  interval 
(0,T ),  they  are  neither  discrete  nor  statistically  independent,.  The 
sample  estimates  are  functionals  of  y(t  )  and  the  only  nontrivial  funotionals 
for  which  -  even  for  a  Gaussian  random  function  -  the  probability  distri¬ 
bution  is  at  present  known  or  can  be  obtained  are  the  linear  integral 
forms  '(weighted  sample  means  )  and  the  quadratic  integral  forms  '(such  as 
mean  square,  correlation  function  and  speotrum  of  the  sample  )? ) 

1)  No  Wiener,  Acta  Math0  5 117,  1930o 

2)  Mo  Kao  and  A„JoFo  Siegert,  J„  Apple  Physics,  ^8,  383,  1947  and 
AnnoMatho  State  18  ,  438,  1947 

3  )  For  Markoffian  random  functions  the  probability  distribution  of  the 
largest  value  in  the  sample  can  be  obtained  (A0JoF0  Siegert,  Phys0 
Rave  31,  617,  1951)  and  for  an  especially  simple  Markoffian  random 
function  the  Iaplace  transform  of  the  distribution  of  the  integral 
of  the  absolute  value  and  certain  other  non-linear  functionals  are 
known  (Me  Kac,  Trans0  Am0  Math0  Soc0  59.  401,  1946;  6£,  1,  1949e) 


The  expectation  value  and  average  square  of  sample  estimates  can  usually 
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be  obtained  for  Gaussian  random  functions,  and  yield  some  information  about 
the  probability  distribution  through  the  Bienaym^  -  Tchebycheff  inequality^ 
.  / 

4;  c„f0  Harald  Cramer,  Itethematical  Methods  of  Statistics,  Princeton 
University  Press,  1946,  pp„  182-183  „ 


Since  most  of  the  above  results  have  been  oaloulated  for  Gaussian  random 


functions  only  it  seemed  of  interest  to  develop  the  basis  for  a  criterion  by 
which  one  may  Judge  whether  at  least  the  first  distribution  of  the  sample 
is  sufficiently  close  to  a  Gaussian  distribution  so  that  the  sample  could 
have  come  from  a  Gaussian  random  funotion0  We  define  the  first  distribution 
$(a,T)  of  a  sample  y(t )  as  that  fraction  of  the  interval  (0,T  )  during  which 
y(t )  >  a0  We  have  computed  the  fluctuation  i?  =  <  |$(a,T  )  -  JJ  > 
and  the  integrals  $  and  £  ^  of  the  fluctuation  over  all  values  of  a 
without  weighting  and  with  a  weight  function  G(a}  rsp  for  Gaussian  random 
functions  with  first  probability  distribution  0( a)  and  with  arbitrarily 
given  correlation  function,, 

The  hypothesis  that  the  sample  came  from  a  Gaussian  distribution  with 

specified  parameters  can  be  rejected  if  the  deviation  0(a) 

at  a  pradetarminad^  ^  valua  a  appreciably  exceed  s  the  value  5,  or  If  the 
roo  2 

integral  J ^  [jP(a,T  )  -  0(a.  JJ  G(a)  da  appreciably  exeeds^jW  S2  rsp0 

5)  "Predetermined"  means  that  the  value  of  a  is  ohosen  without  knowledge 
of  the  sample,  and  not,  for  instanoe,  at  the  point  for  which 
-  ^(a )  |  has  its  largest  value0 

More  precisely  we  know  from  the  Bienayme''-  Tchebycheff  inequality 

f  I  -  t\j  _  \  _/ ,  ,  I  \  .  _  ~1  _ -  _  /.  2 


prob 


-  0(a)j  >  ksj.  <  l/kS 


for  any  predetermined  value  of  a,  and 
prob 


j/^(0(O)  $a,T)-jrf(a]J  da  >  k^(2  )  l  <  l/k 
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There  remains,  of  course,  as  always,  an  arbitrariness  In  choslng  a  value 
for  the  probability  at  which  one  wishes  to  reject  the  hypothetical  distri¬ 
bution,  rather  than  aooept  an  improbable  outoome  of  the  experiments 


1)  Definitions 

The  fraction  of  the  time  T  during  which  y(t )  >  a  is  denoted  by  lP(a,T) 
and  expressed  as 

da) 


i^(«»T)  ~  7  t  PMO  -  *)  dt 


where  p  is  defined  by 

(1.2) 


a/  \  0  _  x  —  0 

P(x>  =  1  for  x >  0 


Not  only  the  problem  of  finding  the  probability  distribution  of  ^(a,T) 


T)  =  1 


but  even  the  special  cases  prob  j^$(a,T )  =  0  -  and  prob 
seem  to  be  extremely  difficult,  except  for  Markoff ian  random  functions. 


fix.. 


sinoe  their  solution  is  closely  related  to  the  solution  of  the  first 

paaaage  time  probability  problems 

a  • 

We  define  the  first  and  second  probability  functions  of  the  stationary 
random  function  y(t)  by 


(1.3) 

prob  <| 

yl  “  y^l  ^  ^yl 

♦  dy: 

and 

(1.4) 

prob  m, 

r 

yl~  y^l^  <yl 

♦  dy 

y2<y(t2)<y2 

♦  dy. 

dy- 
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and  obtain 


T 

<a?(  a,T)>iV  “  <P(y(t)  -  »)>AV  dt 


*(») 


whara 


u 


(1.5) 


<,.)  =  r, 


=  /  W(y )  dy 


la  the  probability  that  y(t )  exeeds  tha  value  a0 

For  tha  second  moaant  we  get. 

T  2 

<iP2(a,Tj>AV  =  <c(\  J  ?(y(t )  -  a)  dtj  >AV 


(1.6) 


.  1 


s  z 

T 


t2  J  J  dtldt2  )  ~a )  P  ))>  AV 

o 

T  oo 

T2  / /dtldt-  / 

o  a 

X  oo 

y1  dt(l  -  ?)  J  J  W2(y1#y2{t )  dyidy2 


2e  Rough  ArDTOTlmatlrnfi 
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.6) 


(2a) 


For  orientation  we  consider  first  the  oversimplified  '  case 

f'y(y1)f(y2'y1)for  ^ 

“t ,  )  —  < 


W2^rl,72,t2“tl )  = 


W(7i  )  W(y.  )  for 


V. 


t2”tl 


> t 


6)  This  is  not  a  realistic  assumption,  since  it  yields  for  the  normalized 
correlation  function 

J°(VV  =  «y(t2)  y(t1)>AV  “<y>AV^<y2>AV  ■  ) 


the  values 

o,t)  .  f1  for  N- 

J  '  “  |  0  for  |  t|  >  V 

This  is  not  a  possible  correlation  function,  since  its  Fourier  trans¬ 
form  can  assume  negative  values  0 


A  simple  geometrio  consideration  yields  for  this  case 

(2.2)  </(., T)>av  =  (1  -  fj2  /(a)  ♦  (1-(1-  *?)/!  (a) 

so  that  the  fluctuation  becomes 


(2.3)  <^(«,T)>av  -  <iJ?(a,T>^v  *  (1  -  (1  -  f?  )(#)(»)-  jfta)) 

»  2^(1  -  £)(/!(*)  -  f(*)) 


As  was  to  be  expected  the  fluctuation  vanishes  for  *  — ^0  and  becomes 
equal  to  0(a)  -  0^(a )  for 

A  rough  idea  of  the  distribution  of  £^(a,T)  can  be  obtained  by  dividing 

the  interval  T  into  m  equal  parts,  ohosing  m  such  that  the  values  of  y(t ) 

approximately 

at  the  centerpolnts  tj  of  the  intervals  can  be  considered  a s/inde pendent 
random  variable s0  A  set  of  Independent  random  variables  is  then 
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P(y(tj  h*) 

defined  such  that  ^  =  0  or  1  if/  *  0  or  1,  respectively. 

The  Man  2^  =  ^  is  an  approximation  for  iP{ a,T) 

Tha  probability  diatribution  for  4^)1 8  then 

(2.4}  prob  2 1  =  (2)  A»)  (1  - 

with  maan  J0(a  )  and  fluctuation  S2  an  -  ff2  ).  This  is  in  agreement 
with  Eq«  (2„3  )  for  V/T  « 1. 


3.  Calculation  of  <<^2(a.Tl>  for  Gaussian  Random  functions 


“  AV 

For  tha  Gaussian  random  function  we  hava 

2 

_  _JC _ 


(3.1}  W(y)  =  -7=^=5-  • 

V  2ircr 


2(7"' 


_  ♦  ^2  * 


(3.2)  W2(yi»y2,t) 


2t ro^V 


2cr2(i-/>2 ) 


To  evaluate  tha  integral  in  Eqe  lo6  va  expand  VL  using  tha  aquation 


7  ) 


2(1-/^} 


(3.3}  » 

n=o 


7)  H.  Cramer,  ref.  4,  p.  133 


and  obtain 


(3.4)  W2(y1,y2,t)  = 


yl  *  y2  y>2Gr^yI) -2/^172 

2CT2  20j2(1-/>2  ) 


2  2 


2ttO^  * 


20"2 


HnVa)Hi(,2/(r)  „ 

A 


nl 


n=o 
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whare  Hn(x )  are  the  He  mite  polynominal«0  With 


(3.5) 


f* 


.  2^ 


•  2  ^  (a/<j-  ) 


=  r\j  2  7T  0(&  )  for  n  =  G 


for  n  >  0 


we  get 


(3.6) 


oo 


2 

2  oo 


W2(y1(y2lt)  dyi<iy2  »  J^*)  ♦  £  •  ^ 

W»1 


2 
L. 

.2 


H^fr/cr) 


ni 


/> 


-rt  » 

Jo  n=l 


=  ^(a )  ♦  e 


cr4 


^  °°  u2 


*d(*/cr ) 


0  n=o 


n| 


n. 

r  dr 


and 


•  *»■ •  i  fvd  • v 


2  1  ♦  r 


2  rT 

(3.7)  S2(.,t'H<a?2(.,T)>Av-<dP(.,I  )>Av  =  ^J  (1"T 

0  0  * 


2 

(j2  ltr 


s2 


oan  be  written  in  terms  of  single  integrals  in  the  form 


(3.8)  S2  = 


At) 

J. 

2tt 


_ A _ 

.  <r2(l*r) 


_  lu.  _ JL 

0-2  IV3 

e  dt 
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Usually,y°(T  )  will  be  so  small  that  the  first  Integral  can  be  negleoted, 
With  the  overclmpllfled  fora  of/3  (t  )i 


(3.9) 


we  get 


(3.10  ) 


yO/t  \  .  J  1  for  I  1 1  S't 
/  ^  *  "  0  f  or  |  t  j  >  V 


■*/ 


o2  1>T 


=  (l-£) 

T  u  2T' 


_i  f1  dr —  e 

2-  Jo 


.  a! 

cr2  x*r 


The  Identity 


o 


la  easily  verified. 


A  convenient  expansion  for  a  <  <  O'  is  obtained  by  changing  to  the 


variable 


(3.11) 


V  - 


V^,/u  ♦  «•) 


and  writing 


(3.12) 


,  O'  (l*r)  _ 


‘  /  A* 

J  !♦  7 

0$ 


.uiizik! 
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w*  get 

(3ol3)  S2(a,T)  = 


Ai  JL  2<r 
ttT  ® 


2  n  t 


(1  *  ^2  J  •rc»ln/>  (1 


(1-  dt 


where  terms  of  order  (* )  have  been  negleoted  under  the  integral, 


To  judge  the  sample  distribution  in  its  entirety  it  may  also  be  useful 
to  compute  from  the  data  the  quantity 


(3ol4 )  £  =  f  j^(«,Tj-0(a) 

J-  cfi  L 


da 


and  compare  it  with  its  expected  valued  We  have 


(3.15) 


/Kt) 


S  «  <  S>Ky  "rtf/1'  dljT 


*-[ 

Jo 


/>(t) 

)<H  f 
Jo 


-*21  f  (I"  J 

T  a/  t t  J  o 


T  VTT 

For  p  (t )  =  e  ^  (Markoff ian  Gaussian  Random  Function)  the  integral  hai 

2(3”  —  0T 

been  evaluated  in  App„  I,  neglecting  terms  smaller  than  — * -  e  ^  0 

PtVtT 


The  result  is 


(3ol6)  fS*  =  —^21  (0c61  _  Co54  (px)"1  ) 

Ptn/tT 


in  this  approximation,, 
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Another  test  which  places  more  emphasis  on  the  deviations  of  tha  sample 


distribution  from  the  Gaussian  at  large  values  of  j  1^  obtained  by  oom- 

o) 

puting  from  the  data  the  quantity  v 

2 

_a _ 

,  «  r—  n 2  2  (T2 

(3<>17)  S2  =  J  ^(a»T)-^(a)  *J2va2  da 

Denoting  by  the  expected  value  of  this  quantity  we  have 


(3^)  & 


■  -  j 

'  "j[  “'“’“I 

T 

« -  (1-  ^ dt  ^ 


2^ 


For  the  Markoff lan  case  we  have  (neglecting  terms  of  order  e  the  result 


(3.19)  ^  =  n'^PT)'1  [j(2)  -  (PT)‘1^(3) 


2 

_  JL  - 


(see  Appendix  III  )  with  £  (2  )  =  J-  =  1065  and  $  (3  )  =  lo20 


8  )  Approximative  formulas  for  the  contributions  to  this  Integral  from 
the  regions  in  vhioh  dP(a,T)  =  0  or  1  are  given  in  Append!*  II, 


4.  Discussion 
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The  results  of  our  calculations  state  in  various  forms  how  far  the  empirical 
first  distributions  of  samples  taken  from  a  Gaussian  noise  with  specified  para¬ 
meters  will  deviate  from  the  true  first  distribution  of  this  Gaussian  noise0 
They  can  thus  be  used  to  judge  whether  a  sample  of  unknown  source  oan  reasonably 
be  assumed  to  have  come  from  a  Gaussian  noise  with  predetermined  parameters c 
Our  results  oan  not  in  all  cases  be  used  to  Judge  whether  the  sample  oan  reason¬ 
ably  be  assumed  to  have  come  from  a  Gaussian  noise  with  parameters  obtained 
from  the  it  self  n  To  obtain  criteria  for  the  latter  decision  would  require 

the  calculation  of  quantities  such  as 


(4.1  J 


—  2 
<  (a>T  )  -  ^(a,u*  > 


AV 


*  « 


where  0(a,p,  (J~  )  is  the  Gaussian  distribution  with  mean  p  and  standard  deviation 
C ,  which  are  both  computed  from  -  and  thus  functionals  of  -  the  sample  function 
y(t  )<,  Expectation  values  such  as  (4«1 )  are  much  more  difficult  to  calculate 
than  our  averages,  and  may  not  be  obtainable  in  a  useful  form0 

An  answer  to  the  second  question  above  can  however  still  be  obtained  from 

our  results,  if  the  size  T  of  the  sample  is  sufficiently  large,  such  that  its 

e  » 

mean  p  and  standard  deviation  (J~  become  reliable  estimates  of  the  true  mean  p 
and  standard  deviation  q~  „  In  the  computation  of  [^(e,T)  -  0(a  jj  ,  £  and  ^2 

from  the  sample  function,  one  can  then  use  various  combinations  of  values  p 

*  * 

and  C~~  deviating  from  p  and  <J~  rBpc  by  their  respective  expected  errors 
(which  are  not  difficult  to  estimate  )0  If  the  various  values  of  <^(a,T)  -  ^(a) 
^5  and  thus  computed  all  lead  to  the  same  conclusion,  our  results  apply 
to  the  seoond  question,,  We  believe  that  the  question  whether  a  sample  can  have 
coma  from  a  Gaussian  noise  can  reasonably  be  asked  only  in  this  case,  when  at 
least  the  estimates  of  the  mean  and  standard  deviation  obtained  from  the  sample 
are  sufficiently  reliable 0 
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AEESKPH  I 


To  evaluate  the  integral 


(1  -V  )  dt 


we  note  that  we  negleot  terms  smaller  than 


f 1  .-!» 


if  we  extend  the 


integration  to  infinity,  since 

i  - 

for  real,  non-negative  t  and  p0  We  then  have 


I  = 


[ 


(1-  dt  *  |(1>  £  J  «'U  (l-^l-e"1*)  dt 

*0 


=  < 


Tax' v  x  p 


nr-i)  _ 


X=o 


f'"1  E*  <p,r1^] 


s=o 


From 


i_  J.  n*4lZsJ 

d*  ^  f(«) 


Oo 


=  2 


y- 

Z_  2i*k 
k=o 


X  =  o 


one  obtains 


+  )  Detailed  derivation  in  RAHD  Corporation  Report  RM-447,  "On  the  Roots 
of  Markoffian  Random  Functions",  A0JoFe  Siegert0 


1 
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n« )/  r(* 4  )  b  v~^2 1-1  #xp 

with  )c  =  2tlg  2 

and  (n-1  )|  (2i  f  (l  -21  n)  $  (n) 

v_  ,  ,  ♦♦  ) 

^  (n  J  ia  the  Riemann  Zefea  funotion  ' 


«•  f 


n=l 


for  n  ^  2  where 


♦♦ )  See  e0g«  Jahnke  Qade,  Tables  of  Functions  p0  266)  ff, 


We  thus  have 


r  {•*£) 


n-l/2  2“1  (i  -  i  ♦  i2  ♦  oco  ) 


and 


-  $$f fei  -  - (1 '  2‘  * 4,2  *  - J 


(i  -  Utj»  ♦  »2 1  <V2-)^)  ♦  ...) 


(2  ♦  1XX )  -  «  (4  ♦  §  OCj-)^  J  ^  ) 


and 


I  =  f1  (2  ♦  DC^  -  (PT  r1  4*2  ^2^1^  ♦  21^i 


,-l 


=  p  N2(i  -  ig  2)  -  (pr) 


-1 


f1^ 


4(1- 1«2]  -  f*  ♦  2(Xg  2  J2 


#61  -.54  OT ) 


r  1 
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Out  aide  of  the  range  of  the  sample,  defined  by  £0(a,T)  =  1  for  all 
a  ^  t  and  A*  ,T )  *  0  for  all  a  ^  u  the  integral  (3<>17)  can  be  obtained 
analytically  as  a  series  in  powers  of  (- cr/l )  and  (cr/u)o  We  have 


Pr  — 12  a‘/2  (J2  P  o  aV2cT 

■S  =  /  l0(a,T)  -  0(a)  * - J  da  ♦  /  0*(O  da 

JtL  J  7  2rra  Ju  A/2^T 


l2/2^ 


1  -  0(a) 


a2/2  O'2 

r  2 
\2rT 


where 


0(a)  = 


V2rrcr2 


l  rr  2CT2 

v'ZF  a  8 


2(7  . 

e  dy 


-x 

e  ax 


v6V2 


valid  for  a  >  >  CT,  and 


a^CT2 


whioh  by  integration  by  parts  and  use  of  the  asymptotic  series  for  the 
error  function  yields  2 


^  ~  (27rf3/2  (CT/uJ3  •  2Cr2 
~  2n~  (ff 


valid  for  u  >  ,>  CT  and 


r  r  -?  a  2/2a2 


r°°  ^2  .a2/2  CT2  , 

X, 


valid  for  CP. 


APFEHDII  in 


P-236 

9-7-51 

-14- 


To  evaluate  the  Integral 
T 

(1  -  t/T)dt  lg(l  -  e-?1) 

we  note  that  the  Integral  oan  be  extended  to  oo  Incurring  an  error  of 
order  e  only0 

We  then  need  -  for  S  =  3  end  S  =  2  -  the  integral 


The  la  t  Integral  is  equal  to 


■-1  dx  .'x  (1  -  .-P1)-1 

Ro  ?( *)*} 


♦  j  E0  Tc  Whittaker  and  Gu  N0  Watson,  Modern  Analysis,  Cambridge, University 
Press,  1935»  Pc  266 


so  that 


dt  1«  (1  -  -P_1^(2J  ♦  P'2  T_1  f  0  ) 


AJFS/hm 


